Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made both neurobiologically more plausible and computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, for example, fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of recurrent neural networks may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics

Quantal Glutamate Release Is Essential for Reliable Neuronal Encodings in Cerebral Networks

Background: The neurons and synapses work coordinately to program the brain codes of controlling cognition and behaviors. Spike patterns at the presynaptic neurons regulate synaptic transmission. The quantitative regulations of synapse dynamics in spike encoding at the postsynaptic neurons remain unclear. Methodology/Principal Findings: With dual whole-cell recordings at synapse-paired cells in mouse cortical slices, we have investigated the regulation of synapse dynamics to neuronal spike encoding at cerebral circuits assembled by pyramidal neurons and GABAergic ones. Our studies at unitary synapses show that postsynaptic responses are constant over time, such as glutamate receptor-channel currents at GABAergic neurons and glutamate transport currents at astrocytes, indicating quantal glutamate release. In terms of its physiological impact, our results demonstrate that the signals integrated from quantal glutamatergic synapses drive spike encoding at GABAergic neurons reliably, which in turn precisely set spike encoding at pyramidal neurons through feedback inhibition. Conclusion/Significance: Our studies provide the evidences for the quantal glutamate release to drive the spike encodings precisely in cortical circuits, which may be essential for programming the reliable codes in the brain to manage wellorganize

How spiking neurons give rise to a temporal-feature map

A temporal-feature map is a topographic neuronal representation of temporal attributes of phenomena or objects that occur in the outside world. We explain the evolution of such maps by means of a spike-based Hebbian learning rule in conjunction with a presynaptically unspecific contribution in that, if a synapse changes, then all other synapses connected to the same axon change by a small fraction as well. The learning equation is solved for the case of an array of Poisson neurons. We discuss the evolution of a temporal-feature map and the synchronization of the single cells’ synaptic structures, in dependence upon the strength of presynaptic unspecific learning. We also give an upper bound for the magnitude of the presynaptic interaction by estimating its impact on the noise level of synaptic growth. Finally, we compare the results with those obtained from a learning equation for nonlinear neurons and show that synaptic structure formation may profit from the nonlinearity

Soft-bound synaptic plasticity increases storage capacity

Accurate models of synaptic plasticity are essential to understand the adaptive properties of the nervous system and for realistic models of learning and memory. Experiments have shown that synaptic plasticity depends not only on pre- and post-synaptic activity patterns, but also on the strength of the connection itself. Namely, weaker synapses are more easily strengthened than already strong ones. This so called soft-bound plasticity automatically constrains the synaptic strengths. It is known that this has important consequences for the dynamics of plasticity and the synaptic weight distribution, but its impact on information storage is unknown. In this modeling study we introduce an information theoretic framework to analyse memory storage in an online learning setting. We show that soft-bound plasticity increases a variety of performance criteria by about 18% over hard-bound plasticity, and likely maximizes the storage capacity of synapses

Evolving Synaptic Plasticity with an Evolutionary Cellular Development Model

Since synaptic plasticity is regarded as a potential mechanism for memory formation and learning, there is growing interest in the study of its underlying mechanisms. Recently several evolutionary models of cellular development have been presented, but none have been shown to be able to evolve a range of biological synaptic plasticity regimes. In this paper we present a biologically plausible evolutionary cellular development model and test its ability to evolve different biological synaptic plasticity regimes. The core of the model is a genomic and proteomic regulation network which controls cells and their neurites in a 2D environment. The model has previously been shown to successfully evolve behaving organisms, enable gene related phenomena, and produce biological neural mechanisms such as temporal representations. Several experiments are described in which the model evolves different synaptic plasticity regimes using a direct fitness function. Other experiments examine the ability of the model to evolve simple plasticity regimes in a task -based fitness function environment. These results suggest that such evolutionary cellular development models have the potential to be used as a research tool for investigating the evolutionary aspects of synaptic plasticity and at the same time can serve as the basis for novel artificial computational systems

Validation of a brief telephone battery for neurocognitive assessment of patients with pulmonary arterial hypertension

BACKGROUND: The effects of pulmonary arterial hypertension on brain function are not understood, despite patients' frequent complaints of cognitive difficulties. Using clinical instruments normally administered during standard in-person assessment of neurocognitive function in adults, we assembled a battery of tests designed for administration over the telephone. The purpose was to improve patient participation, facilitate repeated test administration, and reduce the cost of research on the neuropsychological consequences of acute and chronic cardiorespiratory diseases. We undertook this study to validate telephone administration of the tests. METHODS: 23 adults with pulmonary arterial hypertension underwent neurocognitive assessment using both standard in-person and telephone test administration, and the results of the two methods compared using interclass correlations. RESULTS: For most of the tests in the battery, scores from the telephone assessment correlated strongly with those obtained by in-person administration of the same tests. Interclass correlations between 0.5 and 0.8 were observed for tests that assessed attention, memory, concentration/working memory, reasoning, and language/crystallized intelligence (p ≤ 0.05 for each). Interclass correlations for the Hayling Sentence Completion test of executive function approached significance (p = 0.09). All telephone tests were completed within one hour. CONCLUSION: Administration of this neurocognitive test battery by telephone should facilitate assessment of neuropsychological deficits among patients with pulmonary arterial hypertension living across broad geographical areas, and may be useful for monitoring changes in neurocognitive function in response to PAH-specific therapy or disease progression

A Consensus Definitive Classification of Scavenger Receptors and Their Roles in Health and Disease

Scavenger receptors constitute a large family of proteins that are structurally diverse and participate in a wide range of biological functions. These receptors are expressed predominantly by myeloid cells and recognize a diverse variety of ligands including endogenous and modified host-derived molecules and microbial pathogens. There are currently eight classes of scavenger receptors, many of which have multiple names, leading to inconsistencies and confusion in the literature. To address this problem, a workshop was organized by theUnited StatesNational Institute of Allergy and Infectious Diseases, National Institutes of Health, to help develop a clear definition of scavenger receptors and a standardized nomenclature based on that definition. Fifteen experts in the scavenger receptor field attended the workshop and, after extensive discussion, reached a consensus regarding the definition of scavenger receptors and a proposed scavenger receptor nomenclature. Scavenger receptors were defined as cell surface receptors that typically bind multiple ligands and promote the removal of nonself or altered-self targets. They often function by mechanisms that include endocytosis, phagocytosis, adhesion, and signaling that ultimately lead to the elimination of degraded or harmful substances. Based on this definition, nomenclature and classification of these receptors into 10 classes were proposed. This classification was discussed at three national meetings and input from participants at these meetings was requested. The following manuscript is a consensus statement that combines the recommendations of the initial workshop and incorporates the input received from the participants at the three national meetings

Kidins220/ARMS Is a Novel Modulator of Short-Term Synaptic Plasticity in Hippocampal GABAergic Neurons

Kidins220 (Kinase D interacting substrate of 220 kDa)/ARMS (Ankyrin Repeat-rich Membrane Spanning) is a scaffold protein highly expressed in the nervous system. Previous work on neurons with altered Kidins220/ARMS expression suggested that this protein plays multiple roles in synaptic function. In this study, we analyzed the effects of Kidins220/ARMS ablation on basal synaptic transmission and on a variety of short-term plasticity paradigms in both excitatory and inhibitory synapses using a recently described Kidins220 full knockout mouse. Hippocampal neuronal cultures prepared from embryonic Kidins220−/− (KO) and wild type (WT) littermates were used for whole-cell patch-clamp recordings of spontaneous and evoked synaptic activity. Whereas glutamatergic AMPA receptor-mediated responses were not significantly affected in KO neurons, specific differences were detected in evoked GABAergic transmission. The recovery from synaptic depression of inhibitory post-synaptic currents in WT cells showed biphasic kinetics, both in response to paired-pulse and long-lasting train stimulation, while in KO cells the respective slow components were strongly reduced. We demonstrate that the slow recovery from synaptic depression in WT cells is caused by a transient reduction of the vesicle release probability, which is absent in KO neurons. These results suggest that Kidins220/ARMS is not essential for basal synaptic transmission and various forms of short-term plasticity, but instead plays a novel role in the mechanisms regulating the recovery of synaptic strength in GABAergic synapses

Information Transmission in Cercal Giant Interneurons Is Unaffected by Axonal Conduction Noise

What are the fundamental constraints on the precision and accuracy with which nervous systems can process information? One constraint must reflect the intrinsic “noisiness” of the mechanisms that transmit information between nerve cells. Most neurons transmit information through the probabilistic generation and propagation of spikes along axons, and recent modeling studies suggest that noise from spike propagation might pose a significant constraint on the rate at which information could be transmitted between neurons. However, the magnitude and functional significance of this noise source in actual cells remains poorly understood. We measured variability in conduction time along the axons of identified neurons in the cercal sensory system of the cricket Acheta domesticus, and used information theory to calculate the effects of this variability on sensory coding. We found that the variability in spike propagation speed is not large enough to constrain the accuracy of neural encoding in this system

Pseudorabies Virus Infection Alters Neuronal Activity and Connectivity In Vitro

Alpha-herpesviruses, including human herpes simplex virus 1 & 2, varicella zoster virus and the swine pseudorabies virus (PRV), infect the peripheral nervous system of their hosts. Symptoms of infection often include itching, numbness, or pain indicative of altered neurological function. To determine if there is an in vitro electrophysiological correlate to these characteristic in vivo symptoms, we infected cultured rat sympathetic neurons with well-characterized strains of PRV known to produce virulent or attenuated symptoms in animals. Whole-cell patch clamp recordings were made at various times after infection. By 8 hours of infection with virulent PRV, action potential (AP) firing rates increased substantially and were accompanied by hyperpolarized resting membrane potentials and spikelet-like events. Coincident with the increase in AP firing rate, adjacent neurons exhibited coupled firing events, first with AP-spikelets and later with near identical resting membrane potentials and AP firing. Small fusion pores between adjacent cell bodies formed early after infection as demonstrated by transfer of the low molecular weight dye, Lucifer Yellow. Later, larger pores formed as demonstrated by transfer of high molecular weight Texas red-dextran conjugates between infected cells. Further evidence for viral-induced fusion pores was obtained by infecting neurons with a viral mutant defective for glycoprotein B, a component of the viral membrane fusion complex. These infected neurons were essentially identical to mock infected neurons: no increased AP firing, no spikelet-like events, and no electrical or dye transfer. Infection with PRV Bartha, an attenuated circuit-tracing strain delayed, but did not eliminate the increased neuronal activity and coupling events. We suggest that formation of fusion pores between infected neurons results in electrical coupling and elevated firing rates, and that these processes may contribute to the altered neural function seen in PRV-infected animals